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Background: Interprofessional teamwork is in many ways a norm in modern health care, and 
needs to be taught during professional education. 

Description: This study is an evaluation of a newly introduced and mandatory learning module 
where students from different health profession programs used Improvement of Quality and 
Safety as a way to develop interprofessional competence in a real-life setting. The intention 
of this learning module was to integrate interprofessional teamwork within the students' basic 
education, and to give students a basic knowledge about Improvement of Quality and Safety. 
This report focuses on evaluations from the participating students (n=222), mainly medical 
and nursing students. 

Materials and methods: To evaluate this new learning module, a questionnaire was developed 
and analyzed using a mixed methods design, integrating both qualitative and quantitative 
methods. The evaluation addressed learning concepts, learning objectives, and interprofessional 
and professional development. 

Results and conclusion: A majority of students responded positively to the learning mod- 
ule as a whole, but many were negative towards specific parts of the learning module and its 
implementation. Medical students and male students were less positive towards this learning 
module. Improvements and alterations were suggested. 

Keywords: interprofessional education, quality improvement, IQS, problem-based learning 

Introduction 

Working in interprofessional teams has become more established in today's health care. ' 
Therefore, future health care workers need to practice this during their education. 2 5 The 
Faculty of Health Sciences at Linkoping University, Sweden, has used problem-based 
learning (PBL) and interprofessional learning 1 - 6 as cornerstones in its educational pro- 
grams since 1986. Three interprofessional learning modules, designed to develop higher 
competencies over time, are mandatory for students in all health programs (medicine, 
nursing, physiotherapy, occupational therapy, biomedical analysts). This paper reports 
the students' evaluations of a newly developed learning module in the students' third 
year, when they work in interprofessional teams to practice Improvement of Quality and 
Safety (IQS) in clinical settings. Apart from its theme, the novelty about this learning 
module is twofold: 1) that it is conducted in a clinical setting, and 2) that it is a manda- 
tory interprofessional module for all undergraduate students of the medical faculty. 

Integrating quality improvement in professional health education, as done in 
this module, represents "a new way of viewing students as change agents in clinical 
practice" as they simultaneously learn about the systems and processes that improve 
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safety for patients, and report back to the clinic. 7 In this 
way, the students also function as "vectors" to transfer new 
knowledge about IQS into the clinic. Making all concerned 
professions cooperate is an effective way of performing 
IQS. 8 In this learning module, this was simulated by asking 
students to work in interprofessional groups. 

To be safe and effective future practitioners in health care, 
students need to take in critical competence of IQS. 9 PBL, the 
traditional research process, and systematic IQS all resemble 
the knowledge improvement PDSA (Plan-Do-Study-Act) 
cycle. 10 These processes start with a real problem, and involve 
similar questions and tools. Because of these resemblances, 
it should be easy for PBL-trained students to understand 
IQS." The new learning module evaluated here is tailored 
to train interprofessional cooperation, to prepare students for 
continuous improvement of quality and safety, and to enable 
critical thinking and lifelong learning. 

A few studies already describe interprofessional education 
modules on IQS, but with mainly medical and nursing students, 
and usually as an elective course. 1215 In a study with voluntary 
students, Cusack and O'Donoghue showed that interprofes- 
sional education leads to changes in students' knowledge, 
skills, attitudes and beliefs.' 2 Using self-reflection by a small 
number of students, Robichaud et al found that participating 
in a quality improvement project can be an excellent vehicle 
to promote interprofessional collaboration. 13 Headrick et al 
described multicenter efforts to integrate quality and safety into 
curricula to foster joint learning, but found that few educational 
programs were able to measure changes in students' behaviour, 
changes in organizational practice, or benefits to patients or 
clients. 14 To our knowledge, large-scale student evaluations of 
IQS in a clinical setting, used as an interprofessional learning 
object, mandatory for all undergraduate students of a medical 
faculty, have not previously been reported. 

Aim 

The aim of this study is to describe, evaluate, and discuss a 
new interprofessional learning module on IQS within health 
care. Based on a student questionnaire, we wanted to know 
if the students experienced the module as an effective way 
of learning interprofessional teamwork and IQS. We also 
discuss the results of the evaluation in terms of which student 
groups were most satisfied with their learning, and how the 
course could be improved. 

Description of the evaluated 
learning module 

At the beginning of the two-week learning module on IQS, 
students from different health programs (medicine, nursing, 



physiotherapy, occupational therapy) are formed into small 
interprofessional groups, with an academic supervisor. Each 
group is allocated a clinical ward or primary health care center 
that has advertised an IQS project. A local facilitator introduces 
the group to the improvement area at hand for instance to 
improve the discharge routines at a surgery ward or to advance 
hygiene routines in primary health care. The student group dis- 
cusses and decides on a focus for their improvement project. 

The student group explores the problem area by inter- 
viewing the clinical staff involved and by investigating 
and measuring relevant routines and habits. The students 
then systematize their observations and gained knowledge, 
formulate general questions, and identify learning goals. In 
parallel with this, students participate in mandatory seminars 
and are offered theoretical lectures on IQS. After studying 
theories, students apply their gained knowledge on the chosen 
project. Their reports with suggestions for improvement and 
for measuring the outcome are assessed by the supervisor 
and then presented by the students to the staff of the clinic. 
The staff of the clinic are free to integrate the students' 
contribution into their routines and their everyday quality 
improvement work. 

Materials and methods 

It is common practice within the faculty to distribute evalu- 
ation forms to students at the end of each learning module. 
A new evaluation questionnaire was developed and tested 
and face validated in our research group during a 2-year pilot 
phase before the module was implemented for all students. 
This evaluation questionnaire was handed out to all students 
enrolled in the learning module in spring 2012 (N=248), and 
submitted voluntarily and anonymously. The questionnaire 
was answered by 90% (n=222) of the students. Of these, 53% 
were nursing students, 23% medical students, and 24% were 
students from other health science programs (physiotherapy, 
occupational therapy). Among respondents, 71% were 
between 2 1 and 25 years old and 7 1 % were female. The aver- 
age response rate for the open-ended questions was 48%. 

The questionnaire posed 19 statements to be answered on a 
six-point Likert scale, and three open- format questions. Themes 
addressed were: learning module concept and implementation, 
fulfilment of learning objectives, lectures given, professional 
and interprofessional development, and practice involvement. 
The open format questions asked the students to give examples 
regarding interprofessional group interaction and dynamics, 
and to give additional comments and suggestions. 

A mixed methods design integrating qualitative and quan- 
titative methods was used for data analysis and to compare 
the graded Likert scale answers with the textual data. 16 
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First, we coded the textual answers using conventional 
qualitative content analysis, 17 including structuring the data 
into themes and categories. We also established criteria 
and assigned negative/neutral/positive labels to the textual 
answers according to their content and tone. Analyses of 
the textual answers were done independently by two of the 
authors (KG and CJT) and then merged. To allow for com- 
parison with the quantitative data, we divided our quantitative 
data into the same three categories accordingly. In this, we 
labeled points 1-2 on the six-point Likert scale as negative, 
3—4 as neutral, and 5-6 as positive. We compared quantita- 
tive and textual answers regarding the same themes to scout 
for differences. 

Second, the quantitative answers were simply dichoto- 
mized; ie, points 1-3 in the scale were regarded as 
negative answers, while 4-6 in the scale were regarded 
as positive answers. We analyzed questions regarding 
concept and implementation, learning objectives, reflec- 
tion on professional roles, and perceived effects of inter- 
professional teamwork, and of the IQS project. We also 
analyzed our quantitative data for differences that could be 
attributed to sex or program affiliation. For this we used 
Pearson's one-tailed chi-square test. A P-value of <0.05 
was considered statistically significant. All quantitative 
data were stored in a database and statistically analyzed 
using SPSS 20 (IBM SPSS Statistics, IBM Corporation, 
Armonk, New York, USA). 

Results 

A majority of the students, 69%, reported that their improve- 
ment project had helped them reach the learning objectives of 
the module (Figure 1 ). A majority (82%) also stated that they 
were able to apply methods and tools for their improvement 
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Figure I Responses to the statement, "Our improvement project contributed to us 
reaching our learning objectives." 



40% 




Not at Very 
all much 

so 



Figure 2 Responses to the statement, "I can apply methods and tools in practical 
improvement work." 

work (Figure 2). About two-thirds of the students (64%) 
believed their project would lead to better quality or safer 
care for the patient (Figure 3). 

Answers about professional development in IQS showed 
mixed results. A majority (80%) stated that they had devel- 
oped their professional knowledge and competence about 
improvement work (Figure 4). Sixty-seven percent said 
they had developed their ability to work together with other 
professions (Table 1). 

Analysis of our quantitative data showed differences between 
the students, attributable to program affiliation and sex. Medical 
students rated significantly lower on all the qualitative questions. 
There were no significant differences between the students of the 
other programs. However, on most questions, males rated lower 
than females (Table 1). Overall, 64% of all 222 respondents 
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Figure 3 Responses to the statement, "Our improvement project will lead to 
better care/health for the patient." 
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Figure 4 Response to the statement, "During this learning module, I have developed 
my knowledge and competence about improvement work." 



reported that they believed that their improvement project 
would lead to better care/health for the patient. Two- 
thirds of the students stated that they had developed their 
cooperative abilities; one-third stated that they had not. 

Only 53% felt able to describe and evaluate how their own 
and other's professional knowledge would influence the orga- 
nizations' outcome (Table 1). This aspect was confirmed by 
the open format answers. Only a few students gave examples 
of "how different professional competences contributed to 
the work". Most students attributed the groups' success to 
interpersonal dynamics rather than to professional compe- 
tences. This also echoed the low percentage of students able 
to describe their own interprofessional competence and 
professional knowledge (Figure 5). 



Table I Improvement of Quality and Safety as interprofessional learning 



Chapter/statement 
(abbreviated) 



Answers I not 2 (%) 3 (%) 4 (%) 5 (%) 6 very Difference Program Figure/ 
(n) at all much between affiliation note 

(%) (%) sexes difference 



Learning module concept 

Our improvement project 216 

contributed to us reaching our 

learning objectives 
Learning objectives 

I can apply methods and tools 221 

in practical improvement work 

I can analyze complexity 219 
in change processes 

I can describe my own 220 
interprofessional competence 

I can describe how my own and 222 
other's professional knowledge 
and approach influences the 
organization's outcome 
Professional and interprofessional development 
I had sufficient prior knowledge 218 
to carry out an improvement 
project 

I have developed my 22 1 

knowledge and competence 

about improvement work 

I have developed my ability to 222 

solve problems 



I have developed my ability 
to work together with other 
health care professionals 
Practice involvement 

Our improvement project will 
lead to better care/health for 
the patient 



221 



193 



3.6 



1.4 



2.3 



I 1.3 



I 1.7 



1.4 



3.6 



5.4 



9.0 



6.8 



11.7 13.1 35.6 26.1 



5.0 



6.8 



8.1 



11.3 34.2 32.0 



7.2 



15.8 



19.8 25.2 36.0 8.6 



23.0 17.6 23.9 15.3 8.1 



13.1 22.5 27.0 18.5 7.2 



6.8 18.0 32.0 31.1 9.0 



5.9 10.4 33.3 32.4 14.0 



9.0 18.9 32.0 29.7 5.0 



10.4 14.0 24.3 24.3 



24.3 24.8 



17.6 



14.9 



NS 



Medical < other Figure I 

students 

P=0.0I35 



Male < female Medical < other Figure 2 
P=0.0 1 85 students 
P<0.00l 

NS Medical < other 

students 
P=0.0l I 

Male < female Medical < other Figure 5 
P=0.0 1 85 students 

P=0.0035 
Male < female Medical < other 
P=0.043 students 

P<0.00l 



NS Medical > other 

students 
P<0.00l 

Male < female Medical < other Figure 4 
P=0.0095 students 
P<0.00l 

Male < female Medical < other 
P=0.003 students 
P<0.00l 

Male < female Medical < other 
P=0.0 1 6 students 
P<0.00l 



NS 



Medical < other Figure 3 

students 

P=0.0I3 



Notes: This table describes response distribution and significant differences in response distribution based on sex and educational program affiliation using the Pearson's 
chi-square I -tailed test. N-222 answers (total). 
Abbreviations: n, number; NS, nonsignificant difference. 
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Figure 5 Responses to the statement, "I can describe my own interprofessional 
competence." 

When asked for examples of how the different 
professional competences in their group contributed to 
the project, 46% out of 125 answers were negative, 22% 
were neutral, and 31% were positive. This compares to the 
ratings on "I can describe how my own and other's profes- 
sional knowledge and approach influence the organization's 
outcome", where 25% were negative, 50% neutral, and 26% 
positive. Inability to describe how interprofessional work 
took place seems to echo inability to see how outcome 
was affected by improvement work as an interprofessional 
enterprise. 

Sixty-six students (30%) wrote comments about the 
learning module, providing additional insight. About two- 
thirds of these comments were categorized as negative; the 
rest were neutral or positive. Main themes identified in these 
comments were frustration and anger. Many challenged the 
concept of the learning module and questioned whether 
this was a good way to gain interprofessional competence. 
Criticism about the practical implementation of the learning 
module, irrelevant improvement projects, and insufficient 
cooperation and coordination by the university and the 
involved clinics was also raised by the students. 

To help interpret these responses, we compared the 
comments to students' ratings on statements about the 
same themes, specifically "During this learning module 
I have developed my knowledge and competence about 
improvement work," and "During this learning mod- 
ule I have developed my ability to work together with 
other professions". Combined these questions had 44% 
positive, 41% neutral, and only 14% negative ratings. 
The open format questions thus revealed a more negative 
attitude than the graded questions, although with lower 
response rate. 



Discussion 

Although IQS is a part of many curricula in health 
care education, 1215 not many of these courses have an 
interprofessional design that includes students from all health 
professional programs. Thus, the results and insights from 
this evaluation can be valuable for others designing similar 
learning modules. Our main result from the student evalua- 
tion of this learning module was that the students generally 
believed that they had increased their interprofessional 
competence and their competence to perform improvement 
work. However, we also found that many students could not 
describe how interprofessional teamwork occurred and how 
it contributed to the improvement project. 

Although about two-thirds reported that they had devel- 
oped their problem-solving abilities and ability to work in 
interprofessional teams, students reported that the projects 
did not seem to need all the participants' professional 
competences. In both quantitative and open format answers, 
it appeared that the students did not know what professional 
competence they had and how it was used. 

Several factors may explain these results. The students' 
theoretical knowledge on IQS and their clinical experience 
might have been insufficient at the time they participated 
in the learning module. Medical students rated their 
prior knowledge higher than did other students, but they 
responded less positively on all analyzed statements. Female 
students responded more positively than male students on 
questions regarding the fulfillment of learning objectives 
and professional and interprofessional development. Our 
data seems to be in accordance with the findings reported 
by Wilhelmsson et al, 18 who found that female nursing stu- 
dents were more ready for teamwork and interprofessional 
cooperation. 

The projects were also perceived to revolve too much 
around the nursing profession and were therefore considered 
less relevant to students of other programs, especially of the 
medical program. This may have consequences for other 
students' involvement and feeling of participation in this 
learning module, and can affect their attitudes to interpro- 
fessional work. 

Imbalance in the number of students from the different 
programs gives an imbalance in group composition. This may 
play a role in how interprofessional learning modules are 
experienced as effective or not, and also affects the results 
of the quantitative analyses. 

To resolve the "nursing focus" of the projects, many com- 
ments suggested the use of theoretical improvement projects, 
through multimedia techniques. No longer doing projects 
on site would mean the loss of the practical experience. 
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On the other hand, theoretical projects could be constructed 
as more complex, and relevant for all participating student 
categories. Theoretical projects might also be easier to fit 
in tight schedules, making participation easier. Adequate 
practical implementation, relevant improvement projects, 
and better coordination and cooperation by the university and 
involved clinics/hospitals seem to be crucial for the success 
of the learning module. 

Understanding the concept of the clinical microsystem 19-22 
is seen by many as essential in the planning and execution 
of improvement projects. A clinical microsystem, in short, 
can be explained as the interprofessional environment at the 
ward, clinic, or primary health care center, where the patient 
and family may also be part. Understanding of health care 
as a system, leadership skills, and methods and tools for 
improvement are also important to a successful improvement 
project, 23 and were intended as learning outcomes from this 
learning module. For this, a short visit to the microsystem 
may be insufficient from an IQS perspective, as it does not 
allow full insight into the complexity and optimum function 
of microsystems. 24 

In the learning module, students were asked to choose 
from given problem areas and could not choose independently. 
This could reduce their sense of involvement and ownership 
to the projects. 25 The short time frame also limits possibilities 
for data gathering and feedback. 

Our respondents generally did not question the premises 
behind learning IQS. The students had positive responses 
about the intentions of the learning module despite some 
criticisms about its implementation. Positive as well as 
negative reactions and attitudes to the learning module can be 
of value for its further development, as "much can be learnt 
from disappointments". 26 

Limitations 

This is the first evaluation of this new learning module, and 
further follow-ups of its strengths and weaknesses are neces- 
sary to get more stable results over time. Our questionnaire 
was designed to assess the learning outcomes specific to 
this learning module. Our question about how professional 
competence contributed to the work assumed that there were 
specific profession-related contributions to be recognized, 
which might not be the case. The open format questions had 
modest response rates, and it may be argued that only the 
students with the strongest opinions answered. 

Furthermore, it was not the intention to measure the exact 
knowledge gained by the students. No common postcourse 
knowledge test was conducted, thus we cannot quantitatively 



measure the impact made. Rather, this evaluation measures 
their attitudes and experiences to a course that integrates 
interprofessional learning and IQS. 

Our findings might be useful for learning module 
designers of similar interprofessional education learning 
modules. Our use of mixed methods, we believe, extends 
our understanding and increases this study's reliability and 
usefulness. 

Conclusion and implications 

Our health science students were positive towards learning 
IQS through interprofessional learning. Most students 
reported they had developed their knowledge about IQS as 
well as their interprofessional competence. Some criticism 
was directed towards the nursing focus of many projects 
and insufficient practical implementation and coordination. 
In general, medical students were less positive than other 
students, and female students were more positive than male 
students. 

Practices and clinics engaging students in improvement 
projects may use our results to make their projects relevant 
and challenging to the students. The insights from our study 
indicate that there is a need to think through the complexity 
of the situations presented and the level of interprofessional 
teamwork required to solve the problems. The available 
time frame as compared to the expected outcomes should 
also be assessed before the improvement project is handed 
to the students. 

Students' views on this learning module may serve as 
a basis for the continued improvement of this module, the 
quality improvement scenarios selected, and the evaluation 
questionnaire. Students and their views should be involved 
when designing similar educational modules. 
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